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Abstract 

We use a linear system of Langevin spins with disordered inter- 
actions as an exactly solvable toy model to investigate a procedure, 
recently proposed by Coolen and Sherrington, for closing the hierarchy 
of macroscopic order parameter equations in disordered spin systems. 
The closure procedure, based on the removal of microscopic memory 
effects, is shown to reproduce the correct equations for short times 
and in equilibrium. For intermediate time-scales the procedure does 
not lead to the exact equations, yet for homogeneous initial conditions 
succeeds at capturing the main characteristics of the flow in the or- 
der parameter plane. The procedure fails in terms of the long-term 
temporal dependence of the order parameters. For low energy inho- 
mogeneous initial conditions and near criticality (where zero modes 
appear) deviations in temporal behaviour are most apparent. For ho- 
mogeneous initial conditions the impact of microscopic memory effects 
on the evolution of macroscopic order parameters in disordered spin 
systems appears to be mainly an overall slowing down. 
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The off-equilibrium dynamics of mean field spin-glass models has been a 
central subject of research in recent times f!], 0, 0, |]. Starting from a stochas- 
tic Markovian dynamics for the microscopic variables, it is found that the 
appropriate "order parameter functions" - the spin-spin correlation function 
at different times, and the associated response function - obey non Markovian 
equations. These contain at all times "memory terms" which depend on the 
previous history. It has been recently emphasized that these memory terms 
play an important role at low temperatures, being responsible for asymptotic 
breaking of time translation invariance and aging. 

If one wants to retain a description of the dynamics at a Markovian level, 
it is possible to write down an exact hierarchy of equations which does not 
close. An infinite number of instantaneous "order parameters" is therefore 
required. Recently Coolen and Sherrington ||^, || (CS) proposed a method 
for obtaining a closed set of autonomous macroscopic differential equations, 
by a systematic elimination of microscopic memory effects. The crucial sim- 
plifying hypothesis is that at any time step the system is at equilibrium 
on the surface where some macroscopic variables related to the energy and 
the magetization are constant. They applied this method to the Hopfield 
[0] model and the Sherrington-Kirkpatrick (SK) model, which are the 
archetypical models for attractor neural networks and spin-glasses, respec- 
tively. The CS procedure, by construction, gives the exact equations at t = 
and the correct equilibrium fixed points. In addition it is exact for all times in 
the limit where the disorder is removed. For the SK model and the Hopfield 
model, however, there are no sufficiently detailed and reliable analytical or 
numerical results to allow for a direct test of the procedure at intermediate 
times, although recent data on cumulants of the local field distribution in the 



Hopfield model do suggest deviations between theory and simulations [11 



For the Hopfield model, for instance, non-trivial finite size effects are known 



to persists even in systems of size ~ 10 |12]. Secondly, in both models 



a full analysis following CS requires going through a dynamic version of the 



replica symmetry breaking scheme a la Parisi [T^, for a continuous range of 
times and parameters, which is in practice unattainable. 

Nevertheless, it is worth noticing that even if the CS procedure neglects 
the possibility of aging - the crucial aspect of the spin-glass dynamics - the 
"flow diagrams" obtained for the order parameters in |]^, |[ show qualitative 
agreement with the results of numerical simulations. In order to understand 
the potential and the restrictions of the closure procedure proposed by CS, 
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we present in this letter results of studying an exactly solvable toy model, 
for which replica symmetry in the dynamical equations is stable. Since the 
CS procedure is based on the elimination of microscopic memory effects, this 
also allows us to obtain a better understanding of the role of these effects in 
determining the macroscopic behaviour of disordered spin systems. 

As our toy model we choose a linear system of Langevin spins {cXi} 
with disordered interactions {Jij} and a Gaussian white noise {rii(t)}: 

d ^ 

—ai = Jijaj - ficTi + r]i {Vi{t)Vj{t')) = '2T6ij6it - t') (1) 
at -^^ 

In which the symmetric interactions {Jij} are drawn at random from a Gaus- 
sian distribution with {Jij) = and (Jfj) = JN~2. For large N the eigen- 
value distribution p(A) of the interaction matrix is given by Wigner's semi- 



circular law 14 



p{x) = ^^^^^e[2j + x]e[2j-x] (2) 

so we have to choose fi > 2 J in order to suppress runaway modes. Both 
the statics and the dynamics of this linear model are trivially solved. In 
statics, it is found that the replica symmetric solution is always stable and 
the model does not have a glassy phase. The system is always in a "high 
temperature" condition. Here we can expect that the memory effects play 
only a minor role, compared to spin-glass systems which exhibit a transition. 
The dynamics is solved by transformation to the basis where the interaction 
matrix is diagonal, i.e. ai{t) — >■ crx{t) and r]i{t) — ^ '7a (^) (this transformation 
does not affect the statistical properties of the noise). Alternatively, one could 
write coupled equations for correlation- and response-functions, containing 
memory terms. Following the former route one obtains 

ax{t) = (TA(0)e-*(^-^) + f'ds r/A(s)e(^-^)(^-*) (3) 

Jo 

In particular we can calculate directly the quantitities in terms of which the 
CS procedure will be formulated, the average spin norm Q = j^Y^i and 
the energy per spin E = Y.ij (^iJijC^j + f Ei erf: 

Q{t) = jdX p(A)a2(0)e-2*(^-^) + T j dX^^ [l - e-2*(/^-^)] (4) 
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E{t) = \jd\ p(A)a^(0)(/i-A)e' 



2t(p-X) I ^rp 

2 



(5) 



We will (improperly) call these quantities "order parameters" for the system. 
The macroscopic equilibrium state is found to be 



Q(oo) 



T 

2J2 



p- 



4J2 



-T 



(6) 



We will consider two types of initial conditions. For homogeneous initial 
conditions, cr^(O) = Qo, we can write the solution (^Q) in the compact form 



Q(t) = Qo - 4 







/ ds 


E(s) - -T 


Jo 


. ' 2 . 



E(t) = -T - - 
^ ^ 2 2 



2Jt 



(7) 
(8) 



in which denotes the modified Bessel function |15]. We can use the 

properties of Ii{z) to obtain from (H) directly the short-time and the asymp- 
totic behaviour of the system, for comparison with the results of the CS 
procedure. For short times we find 



Eit) = ^^iQo + t[fiT- Qoifi^ + J2) 



T(p2 + j2) _^gp(^2^3j2 



fiT{fi' + 3J')-Qo{fi^ + Qfi'r + 2J^) +Oit^) (t^O) (9) 



whereas the asymtotic behaviour turns out to be described by 



I g-2t(M-2J) 

^^^^ " 2^ ~ 4v/?(2Jt)3/2 



T - Qo(/i - 2J) + C( 



oo 



(10) 



The second type of initial conditions we consider are inhomogeneous ones, 
where the system is prepared in one specific eigendirection A of the interac- 
tion matrix, so cxf (0) = QoS{X — A)p"^(A). The solution (^H) can now be 
written as 

Qit) = Qo 









- 4 f'ds 


E{s 


)-irl 


Jo 




2 . 


-2t{^l-K) 


It 
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e'^^^t 1-^(4 Jt) 



2Jt 



(11) 
(12) 
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For short times we now find 
Eit) = ^(/i - A)Qo + t[fiT- Qo(/i - A)' 



T{fi' + j')-Qo{fi-Af 



(13) 



whereas the asymtotic behaviour is given by 



E(t) = ir + i(/i-A)goe-^*('^-^) 



rp^-2t{tl-2J) 

4V^(2Jt)3/2 



• OOJ 

(14) 

For n > 2J there are no zero modes and the asymptotic relaxation is sim- 
ply exponential, with characteristic time r = [2(/x — 2J)]~^. For fi = 2J, 
however, zero modes appear, as a result of which we find power laws: 

Q{t) - Q{oo) ~ E{t) - E{oo) ~ 



(15) 



The only asymptotic diffence between the two types of initial conditions is 
that in the case of choosing a zero mode as the initial state (/i = A = 2 J), 
the equilibrium norm Q{oo) will depend on Qq. 

We now turn to the CS procedure [0, ^ for deriving a closed set of de- 
terministic macroscopic differential equations. The total energy per spin is 
separated into two contributions, one of which depends on the realisation of 
the disorder, and one of which does not. These two quantities (or equiva- 
lently any functions thereof) will evolve in time deterministically on finite 
time-scales and are chosen to represent a macroscopic state. For the present 
model we can choose Q and E. From the Fokker-Planck equation associ- 
ated with dll) follows a Liouville equation for the macroscopic probability 
distribution Vt{Q,E), which describes the deterministic flow 



d 
di 
d_ 
di 



Q 



E = ^iT 



E--T 
2 



Q,E;t 



(16) 
(17) 



with the sub-shell average 



Ida pt{a) 5 [Q - Q{a)] 5 [E - E{a)] i Ei IEj J^,(y, + /i^^. 



Q,E;t 



Sda pt{B) 5[Q - Q{a)]5\E - E{a)] 



(18) 



These laws are exact, although not yet closed due to the appearance of the 



microscopic probability distribution pt{ff) in (p^Sl). In the case of the Hopfield 
[^] and the Sherrington-Kirkpatrick model, removing the disorder (by 
putting a = and J = in these models, respectively) closes the hierarchy 
[^, ^. The same happens in the present toy model: for J = the sum over 
sites in (|18|) simply equals fi^Q, the microscopic distribution drops out 



and the equations ([T6| , p^ close. 

Following CS we now close the equations (|l^,|l^) for arbitrary J by assum- 
ing (i) self averaging of the flow with respect to the microscopic realisation 
of the disorder, and (ii) that in evaluating the disorder-averaged sub-shell 
average {h'^)Q,E;t (|T8|) we may assume equipartitioning of probability within 
the {Q, E) sub-shells of the ensemble. As a result {h'^)Q^E;t is replaced by 

_ JdaS [Q-Qia)] S [E-Eia)] ^ [E, J^Ja, + fia,]\ 
^^^Q'^=^ jda5[Q-Q{a)]5[E-E{a)] ^^^-^ ^^^^ 

The set (0,0) is now closed and the sub-shell average (|T^) can be calculated 
with the replica method. From this stage onwards all calculations can be 
performed exactly. The underlying assumptions are guaranteed to be exact 
at t = (upon choosing appropriate initial conditions) and in equilibrium 
(as a result of the Boltzmann form of the equilibrium distribution). 

In calculating (|I^) with the replica method there enters an auxiliary spin- 
glass type order parameter q{Q,E), with the physical meaning 

^ JIdada' j^J:^^^^'^S[Q-QmHE-E{a)]6[Q-Q{a')]6[E-E{a')] 

^ ^ JJdada' 5[Q-Q{a)]5[E-E{a)]6[Q-Q{a')]5[E-E{a')] ^^^^'^ 

(20) 

Upon making the replica symmetry (RS) ansatz one finds three different 
regions in the {Q, E) plane, characterised by different associated values of 
q and of the relevant sub-shell average {h'^)Q^E, the boundaries of which are 
the lines E = Ea and E = Eb'- 

EA = lQif^-J) EB^^Qifi+J) (21) 

By calculating the eigenvalue Aat, which determines the stability of the RS 
saddle point in the so-called replicon direction [ITB], we find that RS is truly 
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stable in the g = region (Aat < 0), and marginally stable in the two g > 
regions (Aat = 0): 



region 


q 


Aat 


dE/dt 


E<Ea 


> 





-2fi{E-lT) + {fi-2J){fiQ 


Ea<E<Eb 





< 


IjT - QJ^ - AE^Q 


E > Eb 


> 





-2^{E-\T) + (/i + 2J)(/ig 



(22) 



In the limit of zero disorder we indeed recover the correct (trivial) evolution 
for the remaining order parameter Q. We will now asses to which degree 
the CS flow equations ([T6|j2^) reproduce or approach the exact results in the 



presence of disorder. 

According to (p!6| , |22|) the flow in the two g > regions is directed into 



the middle region g = 0, where the fixed-point of the flow is indeed given 
by the correct expression (H). At criticality (/i = 2J), however, the g = 
fixed-point is precisely on the regional boundary E = Ea and a degenerated 
stable line E = \T develops in the region E < Ea- Solving numerically the 
flow equations ( [I6|j2^ ) results in figure ^ (in this example /i = 8 and T = 3), 
where we show the flow iterated for < t < 10 from initial states which are 
drawn either homogeneously (on the line E = ^fiQo) or inhomogeneously 
from the extreme modes A = ±2 J (on the lines E = ^Qo{^ ± 2 J), which are 
the boundaries of the physical region). The corresponding flow according 
to the exact equations (^ and (p!2D is shown in figure ^. Away from the 
critical situation = 2J there is a qualitative agreement between the two 
flows, especially for homogeneous initial conditions. For fi = 2J there are 
clear deviations in the region E < Ea (where one finds the zero modes). 
In this picture we have added flow lines starting from initial states with 
E = ^Qoifi — 2J) + e (e <^ 1), to emphasize the difference with the zero 
modes E = jQo{^ — 2J). The degenerated line E = ^^T oi the CS equations 
is in reality not stable. 

If we inspect the temporal behaviour away from the critical situation 
yU = 2 J, we find a remarkable agreement for the case of homogeneous initial 
conditions (see figure |]). In the case of inhomogeneous initial conditions (see 
figure ^) there is a difference between starting from high energy initial states 
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Figure 1: Flow in the (Q, E) plane according to the CS equations, for /i = 8 
and T — 3. Left picture: J/ji — 3/8 (no zero modes), right picture: J/ n = 
1/2 (zero modes). Outer two dashed hues: boundaries of the physical region, 
E = |Q(yU±2J) (note: for J//i = 1/2 one of these coincides with the line 
E = {)). Inner two dashed lines: boundaries E^ b of the g = region. Thin 
horizontal line segment in right figure: degenerated stable line E = |T. 
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Figure 2: Flow in the (Q, E) plane according to the exact solution, ioi fi — S 
and T = 3. Left picture: J/fi = 3/8 (no zero modes), right picture: J/fi = 
1/2 (zero modes). Outer dashed lines: boundaries of the physical region, 
E = i(5(/i±2J) (note: for J//x = 1/2 one of these coincides with the line 
E — 0). Inner dashed lines: boundaries Ea,b of the g = region. 
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Figure 3: Comparison of order parameter evolution starting from homoge- 
neous initial states, for = 8, T = 3 and Jj [i = 3/8 (no zero modes). Solid 
lines: exact equations, dashed lines: CS closure. 
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Figure 4: Comparison of order parameter evolution starting from inhomoge- 
neous initial states, for = 8, T = 3 and Jj [i = 3/8 (no zero modes). Solid 
lines: exact equations, dashed lines: CS closure. 
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A = 2J, where there is again agreement between exact and CS results, and 
starting from low energy initial states A = —2 J, where significant deviations 
occur. Expansion of the flow equations (|r^,^) around the equilibrium state 
gives the leading asymptotic temporal behaviour, which can be compared to 
the exact results (with the dimensionless quantity x = 2J j n): 

ii>2J : E{t)-E{oo) ~ e-*/" Q{t)-Q{oo) ~ e"*/" 
exact : {2fiT)^^ = 1 — x 

(23) 



CS: (2/ir)-i = i(l-x) + iv/l^ 



/i = 2 J : E{t)-E{oo) ~ r"-i Q{t) -Q{oo) ~ 

exact : a = 1/2 

CS : a = l 

The agreement obtained for homogeneous initial conditions, in spite of the 
difference in characteristic relaxation times (p3D , can be explained by study- 
ing the behaviour of log[Q — Q{oo)] and \og[E — E{oo)] as a function of time 
for the exact solution (H). It turns out that the regime of exponential re- 
laxation sets in only extremely close to equilibrium (for \og[Q — Q{oo)] and 
log[i? — -E'(oo)] of the order of 10~^^). For short times one can expand the 
CS equations in powers of t and compare with the exact expansions (|9| JT3|) , 
with the following results: 

^(0) = lnQo : ^Cs(t) = ^cxact(t) + 0{t^) 

(24) 

E{0) = lifi ± 2J)Qo : Ecsit) = ^exact(t) + Oi^) 

which explains the difference in agreement between the two types of initial 
conditions, i.e. between figures ^ and |[ 

The aim of our study was to perform a test of the closure procedure 
proposed in 0, §, which is based on the removal of any memory effects in 
the evolution of macroscopic quantities. For such a test we have chosen a 
model which is exactly solvable, and does not involve the full hierarchy of 
replica symmetry breaking in the dynamical calculations of CS. Our study 
shows that the CS procedure does not lead in any non-trivial case to the exact 
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equations, yet in some cases succeeds at capturing the main characteristics 
of the flow in the order parameter plane. The procedure fails in terms of the 
long-term temporal dependence of the order parameters. However, in absence 
of zero modes, and for homogeneous initial conditions the true asymptotic 
regime turns out to have only restricted relevance, as it sets in extremely late. 
This implies that for homogeneous initial conditions the effect of microscopic 
memory effects on the evolution of macroscopic order parameters results 
mainly in an overall slowing down. 
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